1. Field of the Invention
The present invention relates to an image processing method that performs an image restoration processing of an image.
2. Description of the Related Art
An image (a taken image) that is obtained by an image pickup apparatus contains a blur component of an image caused by each aberration such as spherical aberration, coma aberration, distortion, or astigmatism of an image pickup optical system, and therefore it is deteriorated. The blur component of the image caused by such aberrations means that a light beam emitted from one point of an object is actually spread although it should be ideally collected on one point on an imaging plane again when any aberration does not exist and also any influence by diffractions does not exist, which is represented by point spread function (PSF).
The optical transfer function (OTF) that is obtained by performing the Fourier transform is frequency component information of the aberration, and it is represented by a complex number. An absolute value of the optical transfer function OTF, i.e. amplitude component is referred to as an MTF (Modulation Transfer Function), and a phase component is referred to as a PTF (Phase Transfer Function). The amplitude component MTF and the phase component PTF are frequency characteristics of an amplitude component and a phase component of the image deterioration caused by the aberration, respectively, and the phase component is represented as a phase angle by the following expression.PTF=tan−1(Im(OTF)/Re(OTF))
In the expression, Re(OTF) and Im(OTF) indicate a real part and an imaginary part of the optical transfer function OTF, respectively. Thus, the optical transfer function OTF of the image pickup optical system deteriorates the amplitude component MTF and the phase component PTF of the image, and therefore the deteriorated image is in a state where each point of the object is asymmetrically blurred as it has the coma aberration. Chromatic aberration of magnification is generated because an imaging position is shifted by the difference of an imaging magnification for each light wavelength and it is obtained as color components of for example RGB in accordance with the spectroscopic characteristics of an image pickup apparatus.
As a method of correcting the deteriorations of the amplitude component MTF and the phase component PTF, a method of performing the correction using information of the optical transfer function OTF of the image pickup optical system is known. This method is called an image restoration or an image recovery, and hereinafter, a processing of correcting the deterioration of the image using the information of the optical transfer function (OTF) of the image pickup optical system is referred to as an image restoration processing. As described below in detail, as one of methods of the image restoration, a method of convoluting an image restoration filter that has inverse characteristics of the optical transfer function (OTF) with an input image is known.
In order to effectively perform the image restoration processing, the optical transfer function OTF of the image pickup optical system needs to be precisely obtained. As a method of obtaining the optical transfer function OTF, for example it can be obtained by the calculation based on design value information of the image pickup optical system if the information is provided. Alternatively, it can also be obtained by taking an image of a point light source to perform the Fourier transform for its intensity distribution.
The optical transfer function OTF of the image pickup optical system generally varies in accordance with an image height (a position of an image). Therefore, in order to perform the image restoration processing of the image with high accuracy, an appropriate image restoration filter that is generated based on the variation of the optical transfer function OTF for each image height needs to be used. When characteristics of the image restoration is changed in accordance with the position of the image, it is preferred that the image restoration processing be performed while the image restoration filter is switched in a real space instead of being performed collectively in a frequency space. Japanese Patent Laid-Open No. 2007-183842 discloses a configuration in which an adjustment parameter to adjust a restoration degree of a restored image is provided to continuously change the restoration degree of the restored image.
However, in accordance with the image pickup condition, for at least a part of the image, there is a case where an amplitude component MTF of the optical transfer function becomes zero in a range of Nyquist frequency of an image pickup element. Hereinafter, this is referred to as degradation, and a frequency at which the degradation occurs is referred to as a degradation frequency. As a reason of the degradation, there is an aberration, diffraction, a hand-shake, or the like. Since information related to the degradation frequency are not transferred (not taken), the image restoration processing cannot be performed at the degradation frequency. In addition, when the image restoration processing is performed only at frequencies other than the degradation frequency, ringing is generated at an edge portion of the image. As a result, a high-quality restored image cannot be obtained.
In the configuration of Japanese Patent Laid-Open No. 2007-183842, the adjustment of the restoration degree is possible. However, the configuration of Japanese Patent Laid-Open No. 2007-183842 enables a user to only change the restoration degree, and an appropriate parameter depending on frequency characteristics of an optical system (the optical transfer function) is not automatically determined. Furthermore, since it does not adjust the restoration degree in accordance with the degradation frequency, a difference image caused by the optical system in which the degradation exists cannot be appropriately restored.